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Remarks 



I. 



Status of claims 



Claims 1-51 are pending. 



II. 



Claim rejections under 35 U.S.C. $ 101 



A. 



Determining patentable subject matter under 35 U.S.C. § 101 



In accordance with the "Interim Guidelines for Examination of Patent Applications 
for Patent Subject Matter Eligibility" (published November 22, 2005; referred to herein as the 
"Interim Guidelines"), which clarifies the process described in MPEP § 2106 in view of the 
recent precedential Board decision in Ex parte Carl A. Lundgren , Appeal No. 2003-2088 
(BPAI 2005), the following procedure should be used for determining whether a claim is 
directed to patentable subject matter under 35 U.S.C. § 101 : 



1. Determine whether the claim falls within one of the four enumerated 
categories: process, machine, manufacture, and composition of matter. 

2. Next, determine whether the claim is directed to nothing more than one of the 
following judicial exceptions: abstract idea (such as mathematical 
algorithms), natural phenomena, and laws of nature. In this regard, the issue is 
whether the claim would, in reality, preempt the use of a law of nature or 
abstract idea (e.g., a process that comprises every substantial practical 
application of an abstract idea). The Examiner must identify the abstraction, 
law of nature, or natural phenomenon and explain why the claim covers every 
substantial practical application thereof. If the Examiner meets this burden, 
she has established a prima facie case that the claims are not directed to 
patentable subject matter under 35 U.S.C. § 101. 

3. On the other hand, the claim is directed to patentable subject matter if it recites 
a practical application of any of the three judicial exceptions, which is 
established: 

a. if the claimed invention "transforms" an article or physical object to a 
different state or thing; or 
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b. if the claimed invention otherwise produces a useful, concrete, and 
tangible result. The result is useful if it is specific, substantial, 
credible, and specifically recited in the claim. The result is tangible if 
it is a "real-world result," as opposed to an abstract idea. The result is 
concrete if it can be assured (i.e., the process has a result that can be 
substantially repeatable or the claimed process must substantially 
produce the same result again.). . 
4. If the claim is directed to a practical application of one of the judicial 

exceptions, producing a result tied to the physical world that does not preempt 
the judicial exception, then the claim meets the statutory requirement of 35 
U.S.C. § 101. 

B. The Examiner's position and Applicants' rebuttal 

The Examiner has rejected claim 1-51 under 35 U.S.C. § 101 as being directed to non- 
statutory subject matter. In particular, the Examiner has stated that: 



In claims 1-21, 23-31 and 33-50, the "method" is recited; 
however, all method claimed are abstract ideas not practical 
application, therefore it not limited to tangible, patent-eligible 
subject matter. 

In claims 22, 32 and 51, a "system of organizing" is recited; 
however, system claimed comprises only software components. 
However, it is a computer software per se. 



That is, the Examiner's position is that each of the pending claims falls within one of 
the four categories enumerated in 35 U.S.C. § 101, but the method claims 1-21, 23-31, and 
33-50 are directed to nothing more than abstract ideas and the system claims 22, 32, and 5 1 
are directed to computer software and is therefore ineligible for patent protection. 

With respect to the method claims 1-21, 23-31, and 33-50, however, the Examiner has 
not identified the abstraction to which these claims are solely directed and has not explained 
why each of these claims covers every substantially practical application of that abstraction. 
Therefore, the Examiner has not established a prima facie case that these method claims are 
directed to non- statutory subject matter. 

With respect to the system claims 22, 32, and 51, the Examiner's contention that the 
"system claimed comprises only software components" is not correct. Although the scope of 
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these claims encompasses software components, these claims do not comprise "only software 
components." Moreover, there is no per se rule for computer-related inventions that obviates 
the need for the Examiner to establish a prima facie case of ineligibility for patent protection 
under 35 U.S.C. § 101 (see, e.g., MPEP § 2106 and the Interim Guidelines). Since the 
Examiner has not given any proper reasons for the rejection of these system claims under 35 
U.S.C. § 101, the Examiner has not established a prima facie case that these system claims 
are directed to non- statutory subject matter. 

For at least the reasons explained above, the Examiner's rejection of claims 1-51 
under 35 U.S.C. § 101 should be withdrawn. This rejection also should be withdrawn for the 
following additional reasons. 

1 . Method claims 1-21 

Independent claim 1 has been amended and now recites in part "classifying the 
objects in the sequence to generate a series of object clusters." 

The "classifying" step recited in claim 1 does not involve merely solving 
mathematical problems or manipulating only numbers, abstract ideas or concepts (e.g., a bid 
or a bubble hierarchy) without some claimed practical application; nor do these steps consist 
solely of mathematical operations (i.e., converting one set of number into another set of 
numbers). Instead, in accordance with the ordinary and accustomed meaning of the term 
"classifying" (see, e.g., Merriam- Webster's Collegiate Dictionary, 10th Ed.), the claimed 
"classifying" step involves arranging the objects in classes or assigning the objects to a 
category "to generate a series of object clusters." In addition, whatever "abstract ideas" the 
Examiner believes claim 1 manipulates, there is no reasonable way that claim 1 could be 
considered to cover every substantial practical application of those ideas. 

For at least these additional reasons, the Examiner's rejection of independent claim 1 
under 35 U.S.C. § 101 should be withdrawn. This rejection also should be withdrawn for the 
following additional reason. 

Even if the Examiner can somehow establish that "classifying the objects in the 
sequence to generate a series of object clusters" involves nothing more than manipulation of 
abstract ideas, the invention defined in claim 1 produces a useful, concrete, and tangible 
result (i.e., a series of object clusters). As explained in the specification, the claimed result 
"may be used for any of a variety of applications, including permanent organization, 
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collective annotation, and dynamic visualization" (page 5, lines 24-25). The result is useful 
because it is specific, substantial, credible, and specifically recited in the claim. The result is 
tangible because it is a "real-world result" (i.e., a series of object clusters), as opposed to an 
abstract idea. The result is concrete because it can be assured (i.e., a given embodiment of 
the method defined in claim 1 will produce substantially the same result each time the 
method is executed; in other words, the method defined in claim 1 has a result that is 
substantially reproducible). 

For at least this additional reason, the Examiner's rejection of independent claim 1 
under 35 U.S.C. § 101 should be withdrawn. 

Each of claims 2-22 incorporates the features of independent claim 1 and therefore is 
directed to patentable subject matter for at least the same reasons. 

2. Method claims 23-3 1 

Independent claim 23 recites: 

23. A method of organizing a collection of objects, 
comprising: 

segmenting objects from the collection into clusters; 

extracting context-related meta data associated with the 
objects and parsable into multiple levels of a name hierarchy; 
and 

assigning names to clusters based on the extracted 
context-related meta data corresponding to a level of the name 
hierarchy selected to distinguish segmented clusters from one 
another. 

The "segmenting," "extracting," and "assigning" steps recited in claim 23 do not 
involve merely solving mathematical problems or manipulating only numbers, abstract ideas 
or concepts (e.g., a bid or a bubble hierarchy) without some claimed practical application; nor 
do these steps consist solely of mathematical operations (i.e., converting one set of number 
into another set of numbers). Instead, in accordance with the ordinary and accustomed 
meaning of the terms "segmenting," "extracting," and "assigning" (see, e.g., Merriam- 
Webster's Collegiate Dictionary, 10th Ed.), the claimed "segmenting," "extracting," and 
"assigning" steps involves dividing objects from a collection into clusters, determining 
context-related meta data associated with the objects, and associate names to the clusters. In 
addition, whatever "abstract ideas" the Examiner believes claim 23 manipulates, there is no 
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reasonable way that claim 23 could be considered to cover every substantial practical 
application of those ideas. 

For at least these additional reasons, the Examiner's rejection of independent claim 23 
under 35 U.S.C. § 101 should be withdrawn. This rejection also should be withdrawn for the 
following additional reason. 

Even if the Examiner can somehow establish that the "segmenting," "extracting," and 
"assigning" steps recited in claim 23 involve nothing more than manipulation of abstract 
ideas, the invention defined in claim 23 produces a useful, concrete, and tangible result (i.e., 
object clusters with assigned names). As explained in the specification, the claimed result 
provides "meaningful, context-sensitive names to the object clusters" (page 6, lines 1-3). The 
result is useful because it is specific, substantial, credible, and specifically recited in the 
claim. The result is tangible because it is a "real-world result" (i.e., object clusters with 
assigned names), as opposed to an abstract idea. The result is concrete because it can be 
assured (i.e., a given embodiment of the method defined in claim 23 will produce 
substantially the same result each time the method is executed; in other words, the method 
defined in claim 23 has a result that is substantially reproducible). 

For at least this additional reason, the Examiner's rejection of independent claim 23 
under 35 U.S.C. § 101 should be withdrawn. 

Each of claims 24-31 incorporates the features of independent claim 1 and therefore is 
directed to patentable subject matter for at least the same reasons. 

3. Method claims 33-50 

Independent claim 33 has been amended and now recites: 



33. A method of organizing a collection of objects, 
comprising: 

accessing a sequence of objects segmented into clusters 
each including multiple constituent objects arranged in a 
respective sequence in accordance with context-related meta 
data associated with the objects; 

selecting for each object cluster at least two constituent 
objects representative of beginning and ending instances in the 
corresponding object sequence; and 

in a user interface, graphically presenting the selected 
representative objects of each cluster without graphically 
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presenting representations of unselected ones of the constituent 
objects of the clusters. 



The "accessing," "selecting," and "graphically presenting" steps recited in claim 33 
do not involve merely solving mathematical problems or manipulating only numbers, abstract 
ideas or concepts (e.g., a bid or a bubble hierarchy) without some claimed practical 
application; nor do these steps consist solely of mathematical operations (i.e., converting one 
set of number into another set of numbers). For example, in accordance with the ordinary 
and accustomed meaning of the term "graphically presenting" (see, e.g., Merriam-Webster's 
Collegiate Dictionary, 10th Ed.), the claimed "graphically presenting" step involves 
pictorially showing the selected representative objects of each cluster in a user interface. In 
addition, whatever "abstract ideas" the Examiner believes claim 33 manipulates, there is no 
reasonable way that claim 33 could be considered to cover every substantial practical 
application of those ideas. 

For at least these additional reasons, the Examiner's rejection of independent claim 33 
under 35 U.S.C. § 101 should be withdrawn. This rejection also should be withdrawn for the 
following additional reason. 

Even if the Examiner can somehow establish that the "accessing," "selecting," and 
"graphically presenting" steps recited in claim 33 involve nothing more than manipulation of 
abstract ideas, the invention defined in claim 33 produces a useful, concrete, and tangible 
result (i.e., a graphical presentation of the selected representative objects of each cluster). As 
explained in the specification, the claimed result provides "a user with an intuitive and 
natural user interface for browsing and customizing the automatically segmented object 
clusters" (page 5, line 32 - page 6, line 1). The result is useful because it is specific, 
substantial, credible, and specifically recited in the claim. The result is tangible because it is 
a "real-world result" (i.e., a graphical presentation of the selected representative objects of 
each cluster), as opposed to an abstract idea. The result is concrete because it can be assured 
(i.e., a given embodiment of the method defined in claim 33 will produce substantially the 
same result each time the method is executed; in other words, the method defined in claim 33 
has a result that is substantially reproducible). 

For at least this additional reason, the Examiner's rejection of independent claim 33 
under 35 U.S.C. § 101 should be withdrawn. 
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Each of claims 34-50 incorporates the features of independent claim 1 and therefore is 
directed to patentable subject matter for at least the same reasons. 

4. System claims 22. 32. and 51 

None of the system claims 22, 32, and 51 recites (i) nonfunctional descriptive 
material, such as music, literary works and a compilation or mere arrangement of ideas" (see 
MPEP § 2106 IV.B.l), (ii) functional descriptive material (e.g., data structures or computer 
listings) not claimed as embodied in computer-readable material (see MPEP § 2106 
IV.B.l (a)), or (iii) "nothing but the physical characteristics of a form of energy, such as a 
frequency, voltage, or the strength of a magnetic field" (see MPEP § 2106 IV.B.l(c)). 
Instead, each of these claims is an apparatus claim that recites one or more "engines" that are 
operable to perform specified functions. Accordingly, none of these claims is directed to 
non-statutory subject matter per se, contrary to the Examiner's position (see MPEP § 2106). 

In addition, none of the system claims 22, 32, and 51 involves merely solving 
mathematical problems or manipulating only numbers, abstract ideas or concepts (e.g., a bid 
or a bubble hierarchy) without some claimed practical application; nor do these systems 
consist solely of mathematical operations (i.e., converting one set of number into another set 
of numbers). Instead, each of the system claims 22, 32, and 51 recites more than one of the 
three judicial exceptions, including recitation of a practice application: 

• Regarding claim 22, the claimed "segmentation engine" is operable to classify the 
objects "to generate a series of object clusters." 

• Regarding claim 32, the claimed "segmentation engine" and "naming engine" are 
operable to divide objects from a collection into clusters, extract context-related 
meta data associated with the objects, and assign names to the clusters. 

• Regarding claim 51, the claimed "layout engine" is operable to graphically present 
the selected representative objects of each cluster in a user interface. 

Moreover, whatever "abstract ideas" the Examiner believes claims 22, 32, and 51 
respectively manipulate, there is no reasonable way that these claims could be considered to 
cover every substantial practical application of those ideas. 

In addition, the functional limitations recited in the system claims 22, 32, and 51 
essentially track the pertinent method steps respectively recited in method claims 1, 23, and 
33. Therefore, even if the Examiner can somehow establish that the functional features 



Attorney's Docket No.: 200207387-1 
Amendment dated May 31, 2006 
Reply to Office action dated March 9, 2006 



Applicant : Ullas Gargi 
Serial No. : 10/631,369 
Filed : July 31, 2003 



Attorney's Docket No.: 200207387-1 
Amendment dated May 3 1 , 2006 
Reply to Office action dated March 9, 2006 



Page : 19 of 24 

recited in the system claims 22, 32, and 51 involve nothing more than manipulation of 
abstract ideas, the aspects of the invention defined in these system claims produce useful, 
concrete, and tangible results for the same reasons explained above in connection with the 
method claims 1, 23, and 33. 

For at least these additional reasons, the Examiner's rejection of the system claims 22, 
32, and 51 under 35 U.S.C. § 101 should be withdrawn. 

III. Claim rejections under 35 U.S.C. § 102 

The Examiner has rejected claim 1-51 under 35 U.S.C. § 102(e) over Piatt (U.S. 
2003/0009469). 

A. Independent claim 1 

Claim 1 has been amended and now recites in part "comparing the candidate object 
interval to a weighted measure of cluster extent for the current object cluster," where the 
"candidate object interval [separates] the candidate object from an adjacent object in the 
sequence already segmented into the current object cluster" and "the measure of cluster 
extent corresponding to a current distance spanned by all the objects in the current object 
cluster measured in the selected dimension of the context-related metadata." 

The Examiner has stated that Piatt teaches "segmenting a sequence of objects into 
clusters based on comparisons of successive object intervals to weighted measures of cluster 
extent" in paragraphs 12 and 34-36. The Examiner also has stated that "Piatt teaches 
'wherein measures of cluster extent correspond to spans of recorded generation times over 
which objects in the clusters respectively extend 5 " in paragraphs 12, 37, 41, 45, 58, and 59. 



• In paragraph 12, Piatt merely teaches that that unlabeled data is grouped based 
on a similarity with other data, where a "distance metric may be used to 
identify related media objects." This disclosure does not teach that the 
unlabeled data is grouped based on a comparison of a candidate object interval 
that separates the unlabeled data from an adjacent object in a sequence already 
segmented into a current object cluster to a weighted measure of cluster extent 
of the current object cluster. Indeed, this disclosure does not even hint that the 
unlabeled data is arranged in an ordered sequence with the "other data" nor 
does this disclosure teach that the cluster extent of the "other data" is relevant 
to the grouping process in any way whatsoever. 

• In paragraph 34, Piatt merely describes the sources from which the media 
objects may be supplied and defines the term "cataloging". 



Applicant : Ullas Gargi 

Serial No. : 10/631,369 

Filed : July 31, 2003 

Page : 20 of 24 



Attorney's Docket No.: 200207387-1 
Amendment dated May 31, 2006 
Reply to Office action dated March 9, 2006 



• In paragraph 35, Piatt merely defines the processes of "categorization" and 
"clustering" and indicates that an index may be built by either of these 
processes. 

• In paragraph 36, Piatt merely teaches that media objects in a "collection" have 
similar content or relate to similar subject matter and that "Keywords may be 
used to indicate what information is found in a media object stored under a 
particular collection and/or category or to index a media object." 

• In paragraph 37, Piatt merely teaches that media objects may be cataloged 
based upon correlation of the data for the media object and/or data describing 
the media object (metadata, such as data, time, location) with known data and 
metadata. 

• In paragraph 41, Piatt merely describes an example of the types of ancillary 
data that may be collected during the "data collection" phase of Piatt's 
cataloging method (see ffif 38 and 40). 

• In paragraph 45, Piatt merely provides an overview of his cataloging process. 
In this overview, Piatt explains that the media objects may be processed with 
respect to time, date, and/or location, where the processing may include 
comparing the relevant data with threshold data or performing an inexact 
search. A list of media objects may be pruned based on querying the user to 
determine the most appropriate correlation data. The media object may be 
catalogue based on an inference that was made based on the data for the 
objects in the pruned list. The media object may be indexed in a database 
using the file name of the media object or other known indexing method. 
After the media object is indexed, redundant or useless data is removed. 
Finally, the information for the image is updated to indicate that the image was 
based on an inference. 

• In paragraph 58, Piatt describes an embodiment in which media objects are 
automatically organized into collections by clustering images taken near each 
other in time. This embodiment includes a user interface that shows a user 
one representative image per collection and, in response to the selection of one 
of the representative image, shows the all the images of the collection 
corresponding to the selected representative image. 

• In paragraph 59, Piatt describes the process of automatically organizing media 
objects according to their creation times. In this process, the media objects are 
sorted in ascending order by creation date. A media object is added to a 
current collection by comparing the difference between creation times of the 
media object and the last media object added to the collection to a threshold X. 
In the embodiment described in ^ 59, the threshold X is fixed. In the 
embodiment described in fflf 61-63, the threshold varies adaptively in 
accordance with "a running average of the log time gaps between 
photographs" (f 63). In both of these embodiments, the threshold X is not a 
weighted "measure of cluster extent corresponding to a current distance 
spanned by all the objects in the current object cluster measured in the selected 
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dimension of the context-related metadata," as now recited in claim 1. Indeed, 
the threshold X is independent of the extent of the cluster. 

Thus, none of the sections of Piatt's disclosure that were cited by the Examiner 
teaches "comparing the candidate object interval to a weighted measure of cluster extent for 
the current object cluster," as now recited in claim 1 . In fact, Piatt's cataloging method does 
not involve "comparing the candidate object interval to a weighted measure of cluster extent 
for the current object cluster," as now recited in claim 1. 

For at least this reason, the Examiner's rejection of claim 1 under 35 U.S.C. § 102(e) 
over Piatt now should be withdrawn. 

B. Dependent claims 2-21 

Each of claims 2-21 incorporates the features of independent claim 1 and therefore is 
patentable over Piatt for at least the same reasons. 

C. Independent claim 22 

Independent claim 22 recites features that essentially track the pertinent features of 
independent claim 1 discussed above and therefore is patentable over Piatt for at least the 
same reasons explained above. 

D. Independent claim 23 

Independent claim 23 recites in part "assigning names to clusters based on the 
extracted context-related meta data corresponding to a level of the name hierarchy selected to 
distinguish segmented clusters from one another." 

The Examiner has stated that Piatt teaches the "assigning" step recited in claim 23 in 
ffif 35, 36, 40, and 46. 

• In paragraph 35, Piatt defines the processes of "categorization" in which a user 



predefines labels representing collection, and "clustering" in which all objects 
are placed into collection without using predefined labels. This paragraph, 
however, does not teach anything about naming the clusters based on extracted 
context-related metadata. 

• In paragraph 36, Piatt merely teaches that media objects in a "collection" have 
similar content or relate to similar subject matter and that "Keywords may be 
used to indicate what information is found in a media object stored under a 
particular collection and/or category or to index a media object." This 
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paragraph, however, does not teach anything about naming the clusters based 
on extracted context-related metadata. 

• In paragraph 40, Piatt merely describes the "data collection" phase of his 
cataloging method. This phase involves gathering ancillary data before, 
during or after the capture of a media object. The ancillary data may include 
information added by the user, information from a GPS device, or audio added 
from memory or an external device. This paragraph, however, does not teach 
anything about naming the clusters based on extracted context-related 
metadata. 

• In paragraph 46, Piatt describes a process of using a global calendar to catalog 
a media object. In this process, date information for an image to be cataloged 
is compared to dates in the global calendar. If the date corresponds to January 
1 st , it is inferred that the image was taken on New Year's Day and this 
inference may be used to catalog the image. This paragraph, however, does 
not teach anything about naming the clusters based on extracted context- 
related metadata. 

Therefore, none of the sections of Piatt's disclosure that were cited by the Examiner 
teaches anything about naming the clusters based on extracted context-related metadata, as 
recited in claim 23. For at least this reason, the Examiner's rejection of claim 23 under 35 
U.S.C. § 102(e) over Piatt now should be withdrawn. 

E. Dependent claims 24-3 1 

Each of claims 24-31 incorporates the features of independent claim 23 and therefore 
is patentable over Piatt for at least the same reasons. 

F. Independent claim 32 

Independent claim 32 recites features that essentially track the pertinent features of 
independent claim 23 discussed above and therefore is patentable over Piatt for at least the 
same reasons explained above. 

G. Independent claim 33 

Independent claim 33 has been amended and now recites: 



33. A method of organizing a collection of objects, 
comprising: 



accessing a sequence of objects segmented into clusters 
each including multiple constituent objects arranged in a 
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respective sequence in accordance with context-related meta 
data associated with the objects; 

selecting for each object cluster at least two constituent 
objects representative of beginning and ending instances in the 
corresponding object sequence; and 

in a user interface, graphically presenting the selected 
representative objects of each cluster without graphically 
presenting representations of unselected ones of the constituent 
objects of the clusters. 



The Examiner has stated that Piatt teaches the claimed "selecting" step in Iff! 59 and 

60. 



• In paragraph 59, Piatt merely describes the process of automatically 
organizing media objects according to their creation times. In this paragraph, 
Piatt does not teach anything about selecting for each object cluster at least 
two constituent objects representative of beginning and ending instances in the 
corresponding object sequence. 

• In paragraph 60, Piatt describes selecting a single representative image for a 
current collection. In the only example, provided by Piatt, the representative 
image is the image with the median creation time. 



Therefore, contrary to the Examiner's statement, Piatt does not teach the "selecting" step 
recited in claim 33. 

The Examiner also has stated that Piatt teaches "graphically presenting the selected 
representative objects of each cluster" in ffif 72-77. 



• Paragraphs 72-74 describe steps of the color cluster algorithm shown in FIG. 



• Paragraph 75 describes a method that combines temporal and color clustering 
of photographs. 

• Paragraph 76 describes the graphical user interface shown in FIG. 12. In this 
user interface, a single representative image is shown for each cluster in the 
left pane, and all the images in a cluster corresponding to a selected one of the 
representative images are shown in the right pane. 

• Paragraph 77 describes the graphical user interface shown in FIG. 13. In this 
user interface, the right pane shows thumbnails 1303 of all the user's 
photographs and the left pane shows a single representative image for each 
cluster. When the user selects one of the representative images in the left 
pane, the scrollbar 1304 is scrolled so that the same thumbnail is vertically 
centered in the right pane and highlighted with a colored border. 



10. 
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None of the sections of Piatt's disclosure that were cited by the Examiner, however, 
teaches "in a user interface, graphically presenting the selected representative objects of each 
cluster without graphically presenting representations of unselected ones of the constituent 
objects of the clusters," as now recited in claim 33. In fact, Piatt does not teach such a step 
anywhere in his entire disclosure. 

For at least these reasons, the Examiner's rejection of claim 33 under 35 U.S.C. § 
102(e) over Piatt now should be withdrawn. 

H. Dependent claims 34-50 

Each of claims 34-50 incorporates the features of independent claim 33 and therefore 
is patentable over Piatt for at least the same reasons. 

L Independent claim 5 1 

Independent claim 5 1 recites features that essentially track the pertinent features of 
independent claim 33 discussed above and therefore is patentable over Piatt for at least the 
same reasons explained above. 

IV. Conclusion 

For the reasons explained above, all of the pending claims are now in condition for 
allowance and should be allowed. 

Charge any excess fees or apply any credits to Deposit Account No. 08-2025. 
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